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ABSTRACT 

This report presents the major available evidence to 
date on the validity of the^New Jersey College Basic Skills Placement 
Test (NJCBSPT). The findings are as follows: (1) results of two 
content-validity questionnaires show New Jersey college instructors 
to be in general agreement that the test content is appropriate and 
important to assess; (2) test scores from the NJCBSPT are highly 
correlated with Scores from the Comparative Guidance and Placement 
(CGP) tes^s; (3) the Sentence Structure and Essay tests correlate 
slightly more highly with the Test of Standard Written English (TSWE) 
than with the Scholastic Aptitude Test-Verbal; (4) groups of students 
with different educational backgrounds show large differences in 
average NJCBSPT scores; (5) the percentage of students whom 
instructors thought were placed correctly ranged from 68 percent to 
98 percent; (6) the/ percentage of students who thought they were 
placed correctly ranged from 60 percent to 98 percent; (7) median 
predictive validity coefficients were found to be similar to those 
obtained for other test batteries such as the CGP tests; and (8) 
median concurrent—validity coefficients were found to be as follows: 
reading comprehension, .27; sentence structure, .33; logical 
relationships, ,.36; computation, .48/ elementary algebra, .5*7. 
(Author/GK) 
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^ I. Test Validation 



The purpose of this report is to summarize the major evidence 

to date of the. validity of the New Jersey College .Basic Skills 

Placement Test (NJCBSPT) . The conventional definition of test 

validity is the- extent to which a test measures what it is intended 

to measure, Upon analyzing this definition, it quickly becomes clear 

that validity is not solely a characteristic of a test, but is a 

function of the use to which scores from the test are put. Cronbach 

(1971, p. 447) states the problem well: 

The phrase validation of a test is a source of 
much misunderstanding. One validates, -not a test, 
but an interpretation of data arising from a speci- 
fied procedure « A single instrument is used in many 
different ways—Smith *s reading test may be used to 
screen applicants for professional training, to plan 
remedial ^instruction in reading, to measure the 
effectiveness of an instructional program," etc. ' 
'Since each application is based on a different inter- 
pretation, the evidence that justifies oile application^ 
may have, little relevance to the next. Because every 
interpretation has its own degree of validity, one 
can never reach the simple - conclusion that, a particu- 
lar test "is valid," (italics in original)' v 

Unlike the assessment of various internal qualities of a test, such 

as reliability, difficulty level, and speededness, the determination of 

a test's validity is a multi-faceted process, requiring the collection 

of data from sources external to the test itself. - The validation of the 

NJCBSPT is no exception to this . Given the variety of evidence for the 

validity of the NJCBSPT, it, is very useful to sort the evidence into 

meaningful categories. The framework employed in this report consists 

of three categories of validity: content validity, concurrent/construct 
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validity, and placement validity, Each of these terms will be defined 
in the section of the report that dealsjjith it. It is important to 
note that this classification of validity is somewhat arbitrary, as is 
any other such system, and that a particular kind of evidence may be 
justifiably placed in more than one category. The classification^ 
system used in this report is merely a convenient way of organizing an 
otherwise unwieldy mass of evidence. 
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II. Description of the Test 

All freshmen entering Mew Jersey public colleges are required 
to take the NJCBSPT," which was first introduced in 1978, In addition 
to New Jersey public colleges, a number of private New Jersey colleges 
have chosen to participate in the testing program. The test is 
administered by the participating^ colleges on a schedule determined 
by each college. Students are tested only after they have been admitted 



-to-fche- college; the NJCBSFT is not intended as an admissions test. 

The test consists of five separately-timed multiple-choice 
sections and a 20-minute free response essay question. The five multiple- 
choice sections are as follows: 



Test Section 



Number 




of Items 


Timing 


40 


30 minutes 


35 


30 minutes 


50 


30 minutes 


30 


35 minutes 


30 


35 minutes 



Reading Comprehension^ 
Sentence Structure^ 
.Logical Relation^iips^\ 
Computation 
Elementary Algebra 

Each of these sections is'made up of four-choice items and each is scored 
as number right (i.e., no "correction for guessing !1 >, 

Responses to the 20-ainute essay question are scored at a central 
Essay Reading by two independent readers on a 4-point scale.* Scores 
of zero are reserved for papers that either present an insufficient sample 



Beginning with the third year of testing in 1980, a 6-point essay- 
scoring scale was adopted* All analyses in this report, however, 
are based on data collected prior to the adoption of the 6-point scale, 



of writing or are off-topic. If the two readers'* scores differ by more 
than one point (i.e., 1 vs. 3, 1 vs. 4, or 2 vs . 4), the paper receives 
a third reading and one of the two discrepant scores is adjusted 
accordingly. The reported essay score is the simple sum of the two 
readers' scores, and ranges from 2 &o 8, 

V 

A scaled score is reported fox each of the five multiple-choice* 

r 

sections of the test. The scores were scale4 in 1978 to a mean of 65 
and a standard dfeviation of 10, a scale chosen by the New Jersey Basic 
Skills Council, These scaled scores are equated across the three active 
forms of the test developed to date, , 

In addition %o the five scaled scores, reported for the multiple- 
choice test sections, two composite scores, reported on the same 
scale, are also derived. The Composition score is a weighted average 
of the scores from Sentence Structure, Logical Relationships, and the 
Essay such that each of these receives approximately equal weight in 
the composite. The Total English score is a similarly weighted average 
of the same three scores plus the score from Reading Comprehension: 
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III. Content Validity 

One of the intended purposes of the NJCBSPT is to assess the 
reading, writing, and mathematics skills of entering freshmen in 
'order to assist colleges in the placement of students in courses 
appropriate to their levels of Skills development, The first step 
toward achieving this purpose is to determine those reading, writing, 
ahd mathematics skills that -should v be assessed by such a test, If 
the skills assessed by the test are in fact appropriate to the purpose 
of the test, ^then ^he test is said to have .patent-validity r - It -is - - 
clear from this brief introduction that content validity is characterized 
by two features. First, it is the most fundamental form of validity, 
and second* it is a judgmental, not statistical,* matter. In this 
report, the content validity of the NJCBSPT is assessed in two ways:' " 
- (1) Hy a review of the test construction process, and (2) by an analysis 
of the independent judgments of college instructors not involved in 
the test construction process, 
The Test Construction Process * 

"Rather than test the^^lSlit^ after they are 

constructed, one should ensure validity by the plan and procedures 
of construction." (Nunnally, 1967, p. 80). In this section, the 
method of developing the NJCBSPT will be reviewed. 

The test construction process starts with the formation of two 
test development committees, one £or,reading and writing, and one' 
.for mathematics. These committees comprise faculty members representing 
^the variety of New Jersey colleges: two-year community colleges, 



* 



This section on the test development process is based on contributions 
by Gertrude C. Conlan., 
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four-year state colleges, Rutgers University, the New Jersey Institute 

of Technology, and some New Jersey private colleges. 

The first responsibility of the development committees is to 

determine explicit and detailed specifications of what knowledges and 

skills are to be measured. Once these content specifications have been 

,* • produced^ the committees' work tuVns to writing, reviewing, and) revising 

test items that measure the specified skills. Questions that pass 

- --iairtiAL-conBaittee^revi^w affe put lata pretests , or trial tests, admin- 

** 

istered to students in classes in New Jersey colleges 4 . The performance 

- / 

of the students on these new questions is analyzed; Did students who 
score weli on the test tend to choose the right answer? Did many 
students omit the questions? Is the item too difficult or too easy 
for the students? 

The development committees review the data for individual questions 
collected from the trial testing. If low-scoring students perform/ 
better on a question than do high-scoring students or if many students 
omit the question, it is reviewed again to determine its flaw. Most 

_ likely, such a question^is ambiguous or can be interpreted' so that 

r 1 

more than one answer is correct. Such questions are discarded, as 

are questions that protfe to be so easy that almost 'all students answer 

them correctly or that prove to be too difficult for the specified 

purposes of the test. 

_ . t Then, after the questions that meet both content and statistical 
specifications have been selected for each section, the committee 
reviews the questions again, this time not only to discover problems 
with individual questions but also to judge whether the relationship 

/ 

io- • . 
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among questions in the te£t as a whole "is siSftsf actory . . In the test, 

9 — / 4 

for example, question 14 should not test JJt&ctly the same thing as 

question 15, and question 17 should not provide a clue to the answer to 

question 5. Some questions may be replaced befote this process is 

completed .* * 

Only after the committee has approved each question for accuracy 
and for relevance to the purpose of the test is the test development 
process completed. Assisted by the statistical' data for each_quefi£loxL__ 
and by the experience of its members as teachers, the committee h&e y 
judged .that each of the "sections for which it irresponsible contains 
fair questions '.dealing with appropriate problems ahd is as close as 
possible iiy£ontent and in difficulty level to comparable sections in 
previous test fc^ns. 

Independent Evaluation of the Content of th^MCJCBSPT 

The preceding section of this report describes the process of 
developing the NJCBSPT. A crucial element in that process is the 
setting of content specifications for the test. As a further check 
on the appropriateness of the content of the &JCBSPT, the judgments 
of .college instructors uninvolved with the test development process * 

were collected. This was accomplished by means of two .questionnaires, 

i 

one concerning the reading and writing sections, and the other with 
the mathematics sections, Copies of these questionnaires and the 
"accompanying cover letter are included as Appendix A of this report. 
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The colleges selected for administering the content-validity 
questionnaires were chosen to span the variety of New Jersey public 
colleges. The six participating institutions included two 2-year 
colleges (Essex County College and Brookdale Community College) , two 
4-year colleges (Stockton State College and Kean College) , Rutgers 
University (Livingston College) and the New^Jersey Institute* of Tech- 
nology. Table I contains the number of instructors responding to the 
questionnaire from each of the six .participating colleges. 



Insert Table 1 about here. 



A total of 73 instructors responded to the reading-writing questionnaire; 

— *• 
49 responded to the mathematics questionnaire. 

The design^f -the questionnaires was guided by Bridgeman's (1980) 
study of the validity ^of the Descriptive Test of Mathematics Skills. 
The questionnaires asked teachers of beginning cbllege freshmen to 
make judgments about the series of skills that are intended to be 
measured by the NJCBSPT. Instructors rated each skill as either very 
important, moderately important, or unimportant. The item numbers 
for each skill and a test booklet were provided to each instructor 1 . 
In addition to making judgments about various skills tested in the 
multiple-choice sections of the test, the reading and writing instructors 
were also asked the question, "How important is obtaining a writing 
sample in assessing the writing ab'ility of beginning college students?" 
The results frotti the questionnaires are tallied in Table 2 (reading- 
writing) and Table 3 (mathematics). The tables report the number of ' 



Insert Tables 2 and 3 about here. 
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instructors .rating eack' skill a£ very important (yil, moderately impor- 
tant QQ} and unAmfrortant Qj) , Also ^iven as a m>te succinct summary 
for each, skill are 4 the mean and the standard deviation of the instructor 
responses, with VI»3,MI»2, andU-1, 

* , Responses to the reading -writing questionnaire show Understanding 
main ideas in reading passages" to be judgfd the most important skill 
of those listed - all 73 respondents rated it as 'Very important," The' 
Item eliciting the second greatest amount of favorable sentiment is 
^ that for the written essay. Only three of the instructors rated the ob- 
taining of a writing sample in assessing the 'writing- 'ability of beginning 
College students as less t*ian 'Very important.* 1 There appears to be a 
strong belief among instructors^ of freshman reading and writing courses 

that an actual writing sample is essential to the assessment of writing 

* 

ability. 



The reading-%writing skill that generated the greatest negative 

response was "making analogies in sentences," On the average, however, 

* the * college instructors rated this .skiil as more than moderately 

important (mean of 2.3 on 3-point scale)., 
• . r * 

Responses of mathematics instructors to the second questionnaire 
^showed generally less skill-tp-skill variability than the reading- * " 
writing instructor responses. Only one skill, "solution of linear 
inequalities," elicited less than a majority 'of 'Very important 
ratings, t Each of the mean ratings for the eleven mathematics 
was considerably above the "moderately* important'^category (means 

range t from, 2.4 to 2.9).. 

* ♦ , 
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V 
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In general, the results of' the content-validity questionnaire's 
tend to support the appropriateness of the content of the NJQBSPT 
£^ its intended purpose of assessing the reading, writing, and mathematics 
abilities, of beginning college students. Each of the 22 skills* received 
average ratings higher than "moderately important, 11 with most skills 
receiving average ratings much closer to "very important." These 
findings tend to confirm the judgments of the test development compiittees 

r 

who determined the content specifications for the NJCBSPT. 
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IV/ Concurrent/Construct Validity 

\ 

t 

i 

Concurrent validation is a process of comparing test scores with 
another variable (the "criterion") considered to provide a measure 
of the ability or trait in question. In its purest form, concurrent 
validation is employed when a purportedly "improved" measurement 
procedure (e.g., shorter, less expensive, more convenient) jVjbeing ~ 
proposed as a substitute for an already established procedure,* The 
validity of the new measure is demonstrated by a high Correlation with, 
the established measure, thus justifying :its substitution for the 
.established measure. The more typical application of concurrent valida- 
tion, however, consists simply of demonstrating correlations between 
two tests that are intended to measure the same or a similar trait. It 
la this interpretation of concurrent validity that is drawn upon in the 
section below regarding the relationship between NJCBSPT scores and scores 
derived ftom the Comparative Guidance and Placement (CGP) Program. 

Construct validation is an analysis # of the meaning of test scores 
in terms of psychological concepts or "constructs" (Cronbach and tteehl, 
1955), A test is considered as having a degree of construct validity 
if scores from the test behave in ways consistent with some theoretical 
network of unobservable traits. The kinds of evidence that can 
illuminate the construct validity of a test are particularly diverse. 
Three methods of construct validation will be drawn upon in. this 
report: (1) the demonstration of convergence, or the extent to which 
two tests measure the same thing, (2) the demonstration of divergence, 
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or the extent to which two tests measure different things, arid*. (3? 
the analysis of group differences'. Tt\e* first of these, the demon- 
stration of convergence between testy, is very similar to the notion 
of concurrent validity. In fact, sfcme theorists (e.g. Cronbach, 1970) 
subsume concurrent validity as a special case of construct validity. 
It is because of these overlapping classifications that concurrent 
and construct validity are considered jointly in this r.eport. 
Relationship with CGP Scores 

One source of data bearing on the poncctrrent j(or "convergent") 
validity of the NJCBSPT is the set af scores for 322 students at 
Mercer County Community College who were tested in 1978 with the 
NJCBSPT as well as with the tests in the College Board's Comparative 

Guidance and Placement (CGP) Program. The CGP tests are designed to 

* „ - 

serve purposes similar to those served by the NJCBSPT* The correla- 
tions between analogous tests in the two batteries were as' follows: 
NJ Reading Comprehension and CGP Reading, r * +.75; NJ Sentence 
Structure and CGP Sentences, r = +.73; NJ Computation and CGP Com- 
putation, r - +.67; NJ Elementary*Algebra and CGP Elementary Algebra, 
r « +,72. /The reason for the somewhat lower correlation for the 

Computation test may be that, even though the two tests are identically 
* 

named, the NJCBSPT test contains a substantial number of arithmetic 
reasoning 'problems while the CGP test doe^s not. The relatively high 
correlations between the two batteries of tests suggest that they are 
measuring similar constructs. The CGP tests have been validated 
extensively through th^ CGP Program's own validity study service; the 
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correlations between the already validated CGP tests and the tests It 
the NJCBSPT sugges^ th*t the latter may possess as much validity for 
certain purposes as the CGP tests. 

Relationships with SAT and TSWE Scores * 

In the fall of 1978 a study was undertaken to determine the 
statistical feasibility of using Scholastic Aptitu4e Test (SAT) and 
Test of Standard Written English (TSWE) scores in exempting students 
from faking the NJCBSPT (Hecht, 1978). As part of- that study, 20,427 
students were identified by a computer match who had taken' the NJCBSPT, 
the SAT, and the TSWE. ' The correlations among, the test scores were 
as fpllows: 



4T 



/ NJCBSPT Score 


SAT/TSWE Score 


Correlation 


Reading Comprehension 
Reading Comprehension 


SAT-V 
TSWE 


.74 

' ,68 


Sentence Structure 
Sentence Structure 


SAT-V 
TSWE 


.66 
.75 


logical Relationships 
Logical Relationships 


SAT-V 
TSWE 


.77 
.74. 


Essay , 
Essay 


SAT-V 
TSWE 


.50 
.55 


Computation 


SAT-M 


.72 


Elementary Algebra 


SAT-M 


• 76 



Several things can be noted about these correlations. First, 
the correlations involving' the NJCBSPT Essay score are lower than the 
ot;her figures. This may be due to the relatively low reliability 
of the' essay score, compared to the multiple-choice scores, or to the 
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possibility that the Essay measures something different from the 
multiple-choice sections, A second aspect of these correlations to 
note is their magnitude. They range from .66 to ,77, excluding those 
for the Essay. This suggests a rather substantial element in common 
within pairs of tests, 

A third, and perhaps most informative, aspect of these data are 
the correlations of the two tests in the NJCBSPT intended primarily 
as measures' of writing ability (Sentence Structure and Ess£y) with 
the TSWE, 'which is also intended as a measure of writing ability. 
Both the Sentence Structure test and the Essay correlate more, highly 
with the TSWE than with the SAT-V, despite the lower reliability of 
the TSWE (compared tfo SAT-v}. These differences suggest in a very 
tentative way that these two NJCBSPT tests are taf 'fact measuring a 
unique writing ability trait, in addition of *&urse to ^J-arge general 
verbal ability factor. 
Relationships with Student Background 

A common method of construct validation is to demonstrate that 
groups of persons who would be expected' to differ on some trait 
actually show mean differences on a test that supposedly measures the 
trait in question. As fart of the NJCBSPT, studenta^ar€^asked to 
respond to a series of background questions. For some of these 
questions, one would expect groups of students responding in different 
ways to also exhibit mean test-score differences. The background 
questions analyzed as part of this study concerned (1) number of 
years of high school English, (2) number of years~of high school 
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mathematics, (3) type of high school program (academic, career, or 
general), and (4) self -perception 'of writing ability . Data for these 
analyses came from approximately 44,000 students who took the NJCBSPT 
in 1979. 

The results of these analyses a{e reported in Tables 4-7. Generally 



Insert Tables 4-7 about here. 



large""me£a. differences were observed for groups that differed in 
responses to these background questions. In Tables 4 and 5 a direct re- * 
lationshlp can be seen between years of high school study in English and 
mathematics and NJCBSPT scores. The results in Table" 6 show that stu- 
dents from general and career high school programs exhibit very similar * 
test-score patterns but differ greatly from students from academic pro- 
grams. The findings in Table 7 indicate a substantial relationship 
between students' assessments of their writing ability and their, actual 
reading -writing test scores. Of course this can be seen as a validation 
of students' self -assessment as well as a validation of the test scores. * 

It is important not to interpret these findings in a strictly 
causal way. The variability in these background .factors may not be a 
cause_ of the variability in test scores, but rather the variability in 
both may be explained by some larger underlying variables, such as 
general academic ability and interests. For purposes of construct vali- 
dation, it is necessary only to demonstrate a relationshdfe/between group 
-background differences and .group test-score differences. 
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A Final Note 

It should be pointed out thatj the evidence presented in this section 



for the concurrent/construct validity of the NJCBSPT can^not be con- 
sidered definitive. Construct validity evidence is nearly always tenta- 
tive — it needs to be accumulated over time and from a Variety of 
perspectives. Cronbach (1970 , p. 142) reminds us that "construct validity 
is established through a long -continued interplp&J^etween observation, 
reasoning and imagination . . , The process of construct "validation is the 
same as that by which scientific theories are developed. M Considering 
that the construct validation of a test is a necessarily large under- 
taking, the evidence^ discussed- in this section should be thought as 



first steps in a long process, * * 
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V , Placement' Validity 

The NJCBSPT is designed to be useful as a placement instrument. 
Its major intended purpose is to assess the reading, writing, and mathe- 
matics skills of entering college students to assist in the placement 
of students in courses appropriate to their levels ot skills development. 
Because the*Validity of any test is partially a function of how test 
scores are used, an adequate assessment of the validity of the NJCBSPT 
must include arj evaluation of the test battery as a 'placement' device. 
This section of the report discusses the evidence of placement validity 
for the NJCBSPT. 



It is important at thi>^ql^-tT5^3i^^^iish between the effective- 
ness of a placement test (such as the NJCBSPT) and the effectiveness of 
a placement program . Although the validation of a placement test must . 
take^place in the context of a placement program, the test and the * 
program *are not identical. A carefully conceptualized and well 
managed placement program is necessary in order to adequately evaluate 

a placement test. Put another way, if a placement test is used in an 
»-» » 

inappropriately designed placement program, then there is no way to 
separate the effects of the program from the effects of the test^ and 
the whole, venture may be doomed from thj&^start. Despite t{ie interde- 
pendence of the placement teat^and the placement program, it is important 
to keep^the distinction between tijiem in mind. This report is concerned - 
Vitk thfc validation of the NJCBSPT, not with the evaluation of placement 
programs that utilize the test. 
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♦ Because the proper evaluation of k placement test ami the proper ' 

evaluation of a placement program are so interdependent, the New Jersey 

Basic Skills Council decided to have two studies conducted simultaneously. 

The first of these studies deals with the placement validity of the 

test. The results of that study are presented in this report. The 

second study deals with the evaluation of remedial placement programs 

that utilize scores from the NJCBSPT, This latter study is concerned 

with the reasonableness of the placement process itself and with the 

effectiveness of the program in promoting student growths among other 

things. That study was ^onducted at the Bureau of Educational Research 

and Development in the Graduate School of Education of Rutgers University. 

Its findings are presented in a separate report (Smith and Schavio, 

1980). Although the two studies are summarized in separate reports, 

they were conducted in a coordinated way. Both studies draw on the 

same data base (to be described below), and in fatt many analyses of 

the data bear upon both the validation of the test and the evaluation 

of the placement programs. Por example, the analysis of gain scores 

(post-test scores vs. pre-test scores) is relevant to both endeavors. 

A demonstration of test-score gains as a result of instruction would 

be evidence for .program evaluation (i.e., instructional effectiveness) 

as well as for test validation (i.e., the .test as "sensitive" to basic 

» 

skills^ instruction) , _ • . 

Description of the, Sample * 

Two New Jersey public colleges participated in the joint test- 
validation/program-evaludtlon study, College A* is a two-year county 
community college^ College B is a four-year state college. DaC& at 

* • » V 



these colleges were collected in the fall academic term of' 1979. 
All remedial and developmental courses in reading, writing, and 
mathematics were included in the study. The Sample at College A 
consisted of 1,148 student records; that at College B consi£ted_af * 
.1,261 student records.* The sexual, composition of each of the two 
college samples was 45 percent male and 55 percent female. 

At both colleges, students were placed into remedial/developmental 
courses on the basis of NJCBSPT scores. The placement procedures, ' 
including cut scores, aire specified in detail in the report of the 
placement program evaluation (Smith & Schavio, 1980). College A 
offers six major remedial/developmental courses, two in reading 
(Basic Reading Skills 1 and 2), two in writing (Basic Writing Skills 
1 and 2), and two in mathematics (Basic Math Skills 1 and 2). College 
B offers five major remedial/developmental courses: Introduction to" 
Reading and Study Skills (Reading 090), Developmental Reading and 
Study Skills (Reading 102),' Introduction to Writing (English' 090) , 
Basic Writing (English 103), and Basic Mathematical Skills (Math 090). 
Instructor Satisfaction 

If students are placed into cours.es according to a reasonable 
placement policy and on the basis of scores from a valid placement - 
test, then instructors should find students in their .courses to be 
appropriately prepared to deal with the demands of the course, Under 
ideal circumstances, the resulting composition of students should be 



* 



A student record refers to the data for a student in a course. Thus 
if a particular student is enrolled in two remedial/developmental 
courses, fa© or she accounts for two student records. ' 

i 



fairly homogeneous with none overprepared for the course and none 
underprepar^dT 7 Iff order to obtain an indication of £ow satisfied 
instructors we^e Vith the abilities^of the students placed into their 
courses, the instructors were asked via questionnaire to make the 
following judgment: 

* 

Please estimate the number and percentage of students who 
are underprepared, properly placed, or overprepared in 
your course. 

Number Percent 

Underprepared 
Placed Correctly 
Overprepared 

At College A instructors were asked to respond both toward the beginning 

of the term (less than two weeks into the term) and toward the end of 

the term (in the last two weeks). College B instructors were asked to 

respond only once, which was near the end of the term.* 

The results of this survey are reported in Table 8 for College A 

and in Table 9 for College B. The findings suggest that most instructors* 

♦ 

Insert Tables 8 and**9 about here. 

are generally satisfied that their students have been properly placed, 
although there are some distinct differences among courses. In both 
colleges* mathematics instructors tended to "be less satisfied with 
the composition of their course sections than the reading and writing 
instructors. A striking aspect of these data is the number of 
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students judged to be underprep^red for evea the lowest course offered 
in a sequence. At College A, for example, instructors indicated the 
percent underprepared to be 14 .7 , 10 .5, and 14 .7 in the most basic" 
courses offered in reading, writing, and mathematics, respectively 
(beginning-t>f-term data). Likewise, at College B, 18.4 percent of 
students in the first writing course and 20.4 percent in the first 
math course were thought to be underprepared. Perhaps these findings 
reflect^ more of a dissatisf action-wit^h-the caliper of students in 
general than a dissatisfaction with the way the students were placed 
into courses. 
Student Satisfaction 

A. survey similar to that for course instructors was taken to 
assess students 1 satisfaction with their placement into courses. 
Besides being x ^sked tor evaluate the appropriateness of their om 
placement, students were also asked about the placement^ the other 
students in the class. Specifically they were posed with these two 
questions: 

It In my opinion, I 

A. should have been placed in a lower course. 
B» belong in this course, 

C. should have been placed in a higher course. 

< 

2. In my opinion, most of the other students in this class 

t A. should have been placed in a lower course. 

B. belong in this course. 

C should have been placed in a higher course. 

• i 1 
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The results of these surveys are reported in Table 10 for College A 
and in Table 11 for College B. Several interesting patterns can be 

K 

Insert Tables 10 and 11 about here, 

seen in these findings, especially when compared to the findings from 
the instructor survey. At* College A the students appear to be generally 
more satisfied with their placement than are the course instructors. 
The percentage of students who thought they and their classmates were 
placed correctly ranges from 83,4 (for Basic Math 1) to 97.8 (for 
Basic Reding 1), whil£ these "satisfaction rates" for instructors 
range from 6£,8 (for Basic Math 1) to 94.8 (for Basic Writing 2), 
The trend appears to be just the reverse at College B, where students 
appear to be slightly less Satisfied than their instructors with the 
outcomes of the placement process. 

Another interesting trend at both colleges is that the percentage 
of students who think that , they should have been placed in a higher 
course is invariably higher than the percentage of students who think 
that the other students in the class should have been placed in a 
higher course. (The only exception is students at College A in Basic 
Math 1 where the percentages are equal in *the end-of-term survey.) 
In other words, there is a tendency among these students to feel* that 
they themselves should be in a higher course but that their classmates 
belong where they are. 

• In summary, it must be pointed out that the findings reported 
here regarding instructor^ and student satisfaction certainly cannot 
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be taken as "hard" evidence of the placement validity of the NJCBSPT. 

They are included in this report simply as one more piece of evidence 

bearing l on the overall evaluation of the NJCBSPT. 

Relationships with Course Grades 

One of the most common methods' of validating educational tests 

is the examination of the relationship between test scores and course 

grades (qtf grade point averages). In fact, this is such a common 

technique that correlations between .test scores and grades are called 
validity coefficients. The relationship between scores and grades can 
be ^ predictive (i.e., the test scores are obtained at some point in time 
before instruction begins) ot concurrent (i.e., the testes administered 
at approximately the tjjpme time t^at course grades are assigned).. Both 
predictive and concurrent score-grade correlations were obtained as - 
part of the validation of the NJCBSPT. 

* * ■ r; 

Although the use of- grades as a- Criterion against which to evaluate 
a test has an undeniable appeal, this approach is fraught with methodo- T . 
logical limitations, ■ Tftese limitations have the "effect of diminishing 
or attenuating" "the correlations observed between test scopes and grades 

/One major problem is that such correlations are affected by the 
reliability, ofp the course grades themselves* End-of-«ourse grades 

e assigned by one instructor for performance in one course" at one 
point in time. Furthermore, when different sections of a course taught 
by different instructors are combined for the analysis, any variability 
in the grading standards from instructor to^structor will tend to * 



attenuate the eventual correlations. In addition, the typical scale 
for assigning grades (usually a five-point scale such as A to F) is 
itself a rather rough index, and may therefore contribute to the unreli- 
ability of grades as a criterion measure ♦ Although the reliability of 

** 

grades has not been investigated in this study of the NJCBSPT, >it is 

( 

undoubtedly the case that this less than perfectly reliable criterion 

has tended to attenuate the reported correlations, 

-m ^ > 

A second major factor that may attenuate validity coefficients 
is the placement process- itself . A commonly acknowledged principle 
in cqrrelatiqnal statistics is that the correlation between two vari- 
ables yill tend to be attenuated if the range of values for either 
of t£e variables is restricted. This is a particular concern for validity 
studies conducted on groups of students in particular courses. For 
the two' colleges participating in this Validity study, students were 
placed into courses largely on the basis of their NJCBSPT scores* A 
hatural consequence of tKls process is that students in any particular 
course are likely to be more homogeneous than the student population 
at large. When a validity study is conducted on such relatively 
homogeneous groups of students, the obtained validity coefficients 
can be expected to be at least somewhat attenuated. Thus a validity 

coefficient *of +.30 between^ test scores and grades for a course would 

i 

probably be higher if the correlation were based on a more heterogeneous 

< u 

group* of students. J 
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Besides these methodological limitations, there is another factor 
suggesting that moderate-to-high predictive correlations are not only 
unlikely to occur, but educationally -undesirable . If instruction in a 
subject is highly effective, then variance in end -of -course achievement 
(e.g., course grades) would be expected to be very small, as nearly all 
students would have attained mastery of the subject. In fact, one -educa- 
ttonal theorist CBloom, 1976) has pointed out the inadequacy of any edu- 
cational system in which students' eventual achievement levels are pre- 
dictable from their entering achievement levels. 

With these limitations in mind, the reader is directed to Tables 
12-19, which report the results of the sest-score versus course-grade 
correlational analyses. The first four of 'these tables present the ^ 



Insert Tables 12-19 about here.* 



findings for the reading and writing tests of the NJCBSPT; the latter 
four provide similar information for the two mathematics portions of the 
NJCBSPT . 

The findings for the reading and writing tests can be summarized 
as follows. For the Reading Comprehension .test, the predictive corre- 
lations range from ,23 to *35 across the seven reading^writing courses 
studied at the two colleges. The median of these correlations is .26. 
The concurrent correlations for Reading Comprehension range from .16" to 
.52, with a median of .27. For Sentence* Structure the validity 
coefficients are generally higher. "The predictive correlations range 
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from ".16 to .50, with a median of .34; the concurrent correlations range 
from .28 to .50 with a median of .33. The pattern for Logical Relation- 
ships ia similar; a, median, of ,34 for th& predictive correlations and 
.36 for the concurrent correlations. The median predictive correlation 

for the Essay is .21. The lower coefficient for the Essay may -be due in 

t, 

part to that score's lower reliability, as compared to that of the 

multiple-choice scores. t # 

The findings for the two mathematics* tests show generally higher 

validity coefficients than for the readings-writing tests. An especially 

i 

striking 'aspect of the results is the difference between the- predictive 

and the concurrent correlations . While the two types of correlations 

were of similar magnitude for the reading and writing tests, in the^ase 

of the mathematics tests tjie concurrent correlations are consistently 

higher than the predictive correlations % For the two courses. that had 

both pjredictiv&, and concurrent correlations for the ComputatioQjtest , 

'these correlations are, respectively, .43 and .63 for Basic Math Skills 1 

(College A) and .20 and .33 for/Sath "090 (College B) . Similar differ- 

ences were found for the Elementary Algebra test. The % predictive and 

concurrent correlations are, respectively, ^34 and *63 for Basic Math 

Skills' 2 (College A), and .31 and .51 for Math 090 (College B) . 

In judging the magnitude of these validity coefficients, a con- 

* 

venient frame of reference may be the^-fksults of the great number of 
validity -studies conducted on the tests/of the College Board's Compara- 
tive Guidance and Placement (CGP) Program^ This battery of tests is 
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designed to serve purposes very similar to those intended for the 
NJCBSPT. For 64 validity studies conducted on the CGh Reading and 
Sentences . tests from 1975* to 1978, each of which was based on at 
least 100 students, the median predictive validity coefficients were 
.28 for Reading and ,32 for Sentences, These coefficients are very 
similar to those observed for the NJCBSPT Reading Comprehension and 
Sentence Structure tests. Twenty -nine validity studies were conducted 
on the CGP mathematics tests from 1975 to 1978, also each based on 
at least 100 students. The median predictive validity coefficient 
from those studies was *43, likewise quite similar to the predictive 
validities reported here for the 1 mathematics sections of the NJCBSPT, 
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VI . Summary 

This report presents the major available evidence to date on the 
validity of the New Jersey College Basic Skills Placement Test. The 
findings can be summarized as follows: 

1. Results of two content-validity questionnaires i(one for the 
reading-writing tests and one for ^he mathematics tests) 
show New Jersey* college instructors to be in general agree- 

* ment that the test Content is appropriate and important to 
assess. 'One skill was unanimously judged to be very 
important ("understanding main ideas in reading passages"), 
while others received less positive support (in particular, 
"making analogies in sentences 11 and "solution of linear 
'inequalities") . The written Essay was judged to be very 
important to obtain by all but three of the seventy-two 
respondents. 1 

2. Test scores from the NJCBSPT are highly correlated with 
scores from the Comparative Guidance an4 Placement (CGP) 
tests, a battery that is interided to serve purposes similar ' 
to those of the NJCBSPT. 

3. The Sentence .Structure and Essay tests, ^both intended pri- 
marily as measures of writing ability, correlate slightly 
more highly with the Test of Standard Written English (TSWE) 
than with the SAT -Verbal, suggesting that these tests tap a 
writing -ability component in addition to a large general 
verbal factor. 

* 
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4. ▼Groups of student? with different educatidnal backgrounds 

show large differences in average NJCBSPT scores.^ This is 
not a definitive finding in itself, but it do^s contribute 
to the overall picture of the validity of the^ tests. 

5. Instructors at two intensively studied colleges were generally 
satisfied with results of student placement using the NJCBSPT . 
The percentage of students whom instructors thought were 
placed correctly ranged from 68 percent to 98 percent. 
Students at the same two intensively studied colleges were 

\ 

also ^generally satisfied with the results of their (and their 
classmates 1 ) placement* The percentage of students who* 
thought they were placed correctly ranged from 60 percent to 
98 percent. 

Median predictive validity coefficients (correlations between 
pretest scores and course grades) were found to be as follow^; 
Reading Comprehension, .26; Sentence Structure, .34; Logical 
Relationships, .34; Essay, .21; Computation, .40; Elementary 
Algebra, .31. These* coef f^cietfts are similar to those 
obtained for other test batteries such as the' CGP tests, 
Median concurrent validity coefficients (correlations between 
post-test scores and course grades) were found to be as ■ 
follows: Reading Comprehension, .27; Sentence Structure, .33; 
Logical Relationships, .36; Computation, .48; Elementary 
Algebra, .57. No concurrent coefficients were available ip?*- * 
the Essay, a? that section of the'.test was not included in ' ' 

r* * " 

the post-testing . ' 
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Table 1 



Number of Instructors Responding to 
Content-Validity Questionnaires 



Institution 


' Number 
Responding 
in Reading-Writing 


Number 
Responding in 
Mathematics 


Stockton State College 


8 


7 


Kean College 


16 


10 


Essex County College 


15 


9 


Brookdale Community College 


8 


5 


Rutgers-Livingston College 


13 


•2 


New Jersey Institute of Technology 


13 


16 


Total 


'73 


49 




4 

.34 
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Table 2 

Summary of Instructors' Responses 
to Reading & Writing Content-Validity Questionnaire 

(N-73) 



* 


Frequency 






Skill Description 


VI 


Ml . 


U 


' Mean. 


S.D. 


Reading Comprehension : 












1* Understanding main ideas in 
reading passages 

i> 

w 

2. Understanding direct statements * 
in reading passages 


73 


0 


0 


J. u 


0. 0 


61 


11 


.o- 


2.8 


0.4 


3. Making inferences from reading 


53 


20 


0 


2.7 


0.4 


Sentence Structure; 












4. Using complete sentences 


62 


11 


0 


2.8 


0.4 


5. Using coordination and 

subordination appropriately 


44 


28 


1 


2.6 


0.5 


6. Placing modifiers appropriately 


27 


44 


2 


2.3 


. 0.5 


Logical Relationships: 












7\ Categorizing ideas 


53', 


17 


3 


2.7 


0.5 


8. Using appropriate connectives 
, in sentences 


41 


31 


1 


2.5 


6. "5 


9. Making analogies in sentences 


25 


43 . 


5 


2. 3 - 


0.6 


10. Refeognizing principles of organization 
and relationships among sentences 


,60 


13 


b 


2.8 


0.4 


Essay . • 


' 69 


3 


0 


2.9* . 


- 0.2 



NOTES: VI - Very Important » 3 

MI ■ Moderately Important « 2 
U ■ Unimportant 1 

Not all, instructors responded to each question, 
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Table 3 

^ Summary of Instructors 1 Responses 
to Mathematics Content-Validity Questionnaire 

V 

(N-49) 



• 

Skill 'Description 


Frequency 






VJ. 




u 


Mean 


S.D. 


Computation: ' 












1. Operations with whole numbers 


42 


4 


* 3 


2.8 


0.5 


2. Operations with fractions 


42 


6 


1 


2.8 


0.4 


3. Operations with decimals * 

* * 


44 


4 


1 


2.9 


0.4 


4. Solution of simple word problems 
(involving concepts of percent, 
proportion, and average value) 


41 


8 

r 


0 


2.8 


0.4 


Elementary Algebra: 


• 










1. Substitution in algebraic expressions 


42 


7 


0 


2.9 


QL,3 


2. Translating English phrases into 
appropriate algebraic expressions 


"i 

41 


8 


0 


2.8 


0.4 


3* -Operations involving algebraic expressions 


42 




0 


2.9 


0.3 


4. Operations involving simple polynomials . 


43 


6 


0 


2.9 


0.3 


5. Operations involving exponents and roots 


34 


i> 14 


0 


2.7 


0.5 


6. Solution of linear equations 


44 


5 


0 


2.9 


0.3 


, 7, Solution of linear inequalities 


22 


t 

25 


2 


2.4 


0.6 



NOTES: VI * Very Important - 3 

MI ■ Moderately Important m 2 
0 » Unimportant * 1 



Not all instructors responded to each, quest ion. 

; ' • ." *./; 
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Table 4 



NJCBSPT Scores for Students With Various 
Self -Repor ted Years of High School English 



Test Score 


1 Year 


2 Years 


3 Years 


4 Years 


Reading ' 


Mean 


51.4 


56.8 


59.8 


65.8 


Comprehension 


S.D. 


14.2 


14.4 


13.6 


. 10.8 




N 


931 


2,042 


3,870 


36,812 


Sentence 


Mean 


51.2 


55.8 


59.4 


65.8 


Structure ^ 


S.D. 


12.4 


12.7 


12.5 


10.5 


N 


931 


2,042 


3,869 ' 


36,811 


Logical 


Mean 


51.4 


56.8 


59.6 


65.8 


Relationships 


S.D,. 


13.3 


13.8 


13.2 


10.6 




N 


929 


2,042 


3,867 


36,812 


Essay 


Mean 


"3.64 


4.22 


'4.53 


5.18 




S.D. 


1.46 


1.49 


1.45 


1.26 




N 


921 


2,033 


3,862 


36,887 
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Table 5 



NJCBSPT Scores for Students With Various 
Self-Reported Years of High Softool Mathematics 



Test Score * 


1 Year 


2 Year*—-' 


3 Years 


4 Years^ 


Computation 


Mean 
S.D. 

. N 


55.4 
10.2 
2,780 


59.0 
10.3 
9,233 


64.3 
10.3 
15 ,599 


68.0 
10.3 
15,673 


Elementary 
Algebra 


Mean 
S.D. 
N 


54.4 
6.5 
1,772 


56.8 
7.4 
7,438 


64.1 
10.2 
14,334 " 


• 71.0 
11.8 
14,588 



1 
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Table 6 

NJCBSPT Scores for Students from 
Academic, General, and Career High^chool Programs 



Test Score 


Academic 
H. S. Program 


General 
H. S. Program 


Career 
H. S. Program 


.Reading 


Mean 


57.5. 


59.1 


59.1 


Comprehension • 


S.D. 


. 9.8 . 


13.3 


* _ 

13.3 




N 


29,991 


8,958 


7,oBi 


Sentence 


Mean 


67.6 


58.5 


59.2 


Structure. 


S.D. 


9.6 


12.0 


12.2 




N 


29,991 


8,954 


7,052 


Logical 


Mean , 


67.5 


58.9 


59.2 


Relationships 


S.D. 


9.6 


12.7 


13.0 




N 


29,989 


8,952 


7,052 


Essay 


Mean 


5.34 


4.46 


4.62 




S.D. 


1.21 


1.39 


• 1.44 




N 


30,044 


8,947 


7,058 


Computation 


Mean 


67.7 


58.2 


i 

58.0 




S.D. 


9.8 


11.0. 


10.8 




N 


29,896 


8,907 


7,035 


Elementary 


Mean 


68.8 


58.5 • 


56.4 


Algebra ' 


S.D. 


11.6 


9.5 


8.3 " 




N 


28,882 


6,788 


4,833 



Table 7 - 



NJCBSPT Scores for Students Who Reported 
Being Below Average, Average, and Above Average 
; in Written Expression 



.Test Score 


Below 
Average 


Average 


Above 
Average 


Reading 


Mean 


55.8 


62.1 - 


•67.5 


* Comprehension 


S,D. 


13.7 


11 .8 


10.9 


- * 

\ 

Sentence " 


N 


2,375 


20,620 

' '% 


23,460 


Mean 


55.0 


'62.0 


67.5. . 


Structure 


S.D. 


12.1 


11.0 


10 „6 




N . 


2,374 


20,617 


_ 23,460 


Logical' ■ ^ 


Mean 


*' 55, y 


62.1 . 


67.4 


Relationshf^fT* ' 


S.D." 


"*13.1 


> 11 14 


10.7 


N 


2,374 


20,615 


23,458 


Essay 


Mean 


3.92 


' 4776 


5.-40- 4 - 




S.D. 


,1.40 


1.25 


1.30 




N 


, -2,348 


20,658 


23,486 



40 



6 
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Table 8 




Summary of Responses' to Instructor Satisfaction 
Questionnaire at College A 



Course 


'Beginning or ^ 
End of Semester 


1 

Number of 

7 > w UUUCULq 

Represented 


Percent 
Underprepared 


1 Percent 
t- Placed 
Correctly 


Percent 
Overprepared 


28-03 

Basic Reading 
Skills 1 


Beginning (11) 


204 


14.7 


80.9 


4.4 


End (3) 


41 L 


» 14.6 " ; 


; 73.2 ' 


12.2 ' 


28-04 

Basic Reading 
Skills 2 -* 


"Beginning (a) 


• 152' 

l i A— 


'7.2 


> 83.6 


9.2 


End (8) 

- - — ' 1 i - 


137 


■ 8.0 

» 


• 8^.8 


2,2 


28-07 

Basic Writing 
Skills, 1 ' 

** 


Beginning (7) 


133 


10.5 


88'. 7 "■ 


0.8 


^ End -(7) 


122 


8.2 


89/3 C/V 


2.5 


28-08"' 

Basic Writing 
Skills 2 

7 


'^.Beginning (6) 
— — * 


- 'US 


3.5 


'94.8 




1.7 ' 


End (5) • 


. ioo 


21.0 


74.0. . 


5.0 


52-01 

Basic Math 
Skills, 1 


Beginning (21) 


'3,88 


. 14 ."7 7 


- 74.0 


" 11.3 


End (8) ■". 


118. ' 


16 -1 


67^8 


16.1 


52-02 

Basic Hath - 


Beginning- (21) 


J A 

; • 354 


23.7 


68.9 


7.3 


Skills 2 
* 


End (12) , 


228 


. *22.8 


. 70.6^ 


. 6.6 



Number of responding -sections is given in parentheses. 
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fable 9 

Summary of Responses to Instructor Satisfaction 
Questionnaire at College B 



* Course 


Number of 
Sections 
Responding 


Number ot 
Students , 
Represented 


Percent 
Underprepared 


percent . 
Placed 
Correctly 


Percent 
Overprepared. 

1 


Writing 090 


2 


38 


18.4 


' ' 71.1 


10/5 * 


Writing 103' 


21 


383 


9.1 


88.3 


2.6 . 


Reading 090 


2 


43 


0.0 


97.7 


2.3 


Reading 102 


12 


'213 


9.4 


76.5 


M.l 


Math 090 


21 


387 


20.4 


76.5- 


3,1 , 
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Table 10 



Summary of Responses to Student 
Satisfaction Questionnaire at College A 



Course 



Placement 
of Self 
or Others 



Beginning or 
End of Semester 



Number of 

Students 

Responding 



Percent 
Lower 
Course 



Percent 

This 
Course 



Percent 

Higher 

Course 



28-03 
Basic Reading 
Skills 1 



Self 



Beginning 
End 



97 
51 



Others 



Beginning 
End 



93 
49 



5.2 
0.0 



88.7 
92.2 



2.2 
6.1 



97.8 
91.8 



6.2 S 
7.8 



0.0 
2.0 



28-04 
Basic Reading 
Skills 2 



28-07 
Basic Writing 
Skills 1 



Self 



Beginning 
End 



121 
76 



Others 



Self 



Beginning 
End 

Beginning 
End 



119 
76 

70 
105 



Others 



Beginning 
End 



65 
99 



0.0 
0.0 



87.6 
88.2 



0.0 
1.3 

1.4 
1.9. 



93.3 
90.8 

84.3 
87.6 



4.6 
4.0 



95.4 
91.9' 



12.4 
11.8 



6.7 
7.9 

14*. 3 
.10.5 



0.0 
4.0 



28-08 
Basic Writing 
Skills ' 2 



52-01 
Basic Math 
Skills 1 



Self 



Beginning* 
End 



130 
97 



Others 



SeLf 



Beginning 
End 

Beginning 
End 



124 
95 

139 
69 



Others 



Beginning 
End 



134 
69 



0.8 
1.0 



87.7 
81.4 



0.0 
1.1 

2.2 
2.9 



93.5 
88.4 

84.2 
91.3 



3.0 
5.8 



93.3 
88.4 



11.5 
17.5 



6.5 
10.5 

13.7 
5.8 



3.7 
5.8 
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'Table 11. 

Summary of Responses to Student 
Satisfaction Questionnaire at 'College B 



Course 


Placement 

of Self 
or Others 


Number 

of Students 
* 

Responding 


Lower ' , 
Course 


Pompon f* 

This 
Course 


.Higher 
Course 


Writing 090 


Self 




^ 0.0 . 


60.0 


40.0 


Others 


20 


0r0 


80.0 


~20.Q 


Writing 103° 


Self 


- 253* 


0.0 


,81.0 


19.0 


Others 


251 


1.6 


87.6 


10.8 


Reading 090 


Self 


.21 ' 


• 0.0 


66.7 


33.3 


Others 


21 


0.Q 


85'. 7 


14.3 


Reading 102 


-Self 


179 


1.1 


64.2 


34.6 


Others 


175 


1.1 


74.3 


24.6 



* 

Responses from the sections of Math 090 were too fragmentary to include 
, in the analyses. 
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Table 12 

Summary of Predictive Score-Grade Correlations for 
Reading and Writing Courses at College A 



fc 

Course 


Reading 
Comprehension 
Score 


Sentence 
Structure 
Score 


Logical 
Relationships 
Score 


Essay 
Score 


Course 
Grade 


Multiple 
Correlation 


28-03 

Basic Reading 
Skills 1 
(N » 106) 


Mean 4 2. .3 
S.D. 6.8 
r +.293 


48.0 
8.7 , 
+.495 


44.8 
8.4 
+.464 


3.37 
1.21 
+.399 . 


2.29 
1.15 


.565 


28-04. 

Basic Reading 
Skills 2 
(N =« 102) • 


'Mean 57.1 
S.D. 6.0 
r . +.226 


57.0 
8.2 , 
+.236 


57.2 
7.6 
+.256 


4.39 
.91 
+.097 


2.90 
.74 


.325 


28-07 

Basic Writing 
Skills 1 
(N - 128) * 


Mean 46 . 4 
S.D. 10.3 
r ' +.245 


46.6 
7.2 
+.164 ■ 


46.3 
9.3 
+.272 


3.15 
1.00 
+.155 


1.79 
1.32 


.283 1 


28-08 

Basic Writing 
Skills 2 

Or---- 113 > 


Mean 58.5 
S.D. 9.1 
r +.263 


57/1 
8.2 
+.303 


58.0 
8.8 
+.218 


4.14 
.94 
+.214 


2.35 
' 1.14 


.360 


4 

P 


a . 




* 

* 










1 
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* 


f 



\ 
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Table 13 



Summary of Concurrent Score-Gra/ie ^Correlations for 
Reading and Writing Courses at College. A 



Course 


Comprehension 
Score 


Structure 
Score 


LnoM cal 
Relationships 
Score 


Course 
Grade 


Multiple 
Correlation 


28-03 
Skills 1 


Mean . 47,4 
S*D* 10*2 
Number 95 
r +.515 


i 

51.5 
10.0 
67 
+.503 


• 

48.8 
10.2 
* 79 
+.606 


2.42 
1.03 
48 


48 
.624 


28-04 

Basic Reading 
Skills 2 


Mean 61.2 
S.D. 6.7 
Number 96 
r +.210 


59.6 
9.3 
47 
+.282 


• 61.4 
7.2 
93 
+,-275 


"3.04 
.78 
47 


47 
.310 


28-07 

Basis Writing 
Skills 1 


Mean. 48.1 
.S.D.' 10.0 
Number 92 
r +."160 


51.1 
9.3 
104 
+.286 


t 

47.3 
» 9.6 
83 
+.260 

< 


2.54 
.68 
48 


48 
.331 


28-08 * 

Basic Writing 
Skills 2 


Mean 59.5 
S.D. 8.6' 
Number 51 
r +.270 


58.4 
10.2 
58 
+.294 


■ .£3 

60.3 
8.6 , 
47 
+.362 


2.68 
.90 
25 


25 
.436 



. 4' 



<r*> 
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Table 14 / 

Summary of Predictive Score^rade Correlations for 
Reading and Writing Courses at College B* 



Course* 


•Reading 
Comprehension 
Score 


Sentence 
Structure 
Score 


Logical 
Relationships 
Score 


Essay 
Score 


Course 
Grade 


Multiple, 
Correlation 


Writing 090 
(N = 18) 


Mean 54.' 9 
S.D. 14.5 
r +.346 


57.2 
12.8 
+.472 


55.8 
13.0 
+.522 


3.61- 
1.10 
-.037, 


.58 
.50 


.709 


Writing 103 
(N =» 423) 


Mean 66.9 
SiD. 9.1 
r +.254 


66.9 
8.8 
+.390 


66.-3 
9.0 
+.342 


5.05 
l.„00 
+."299 


2.15* 
1.06* 


.446 


Reading 102 
251) 

* 


Mean 61.1 
S.D. 8.5 
r +.285 


62.2 
8.3 
+.343 


60.8 
8.5 " . 
+.409 


5.04 
1.08 
+.386 


2.56 
1.15 ' 


.482 



Reading 090 is not included because no students received an unsatisfactory grade CO) in the 
course, * ' < 
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' Table 15 

Summary of Concurrent Score-Grade Correlations for 
Reading and Writing Courses at College B 



* 

vUU 1. 3C 


Reading 
Scqre 


Sentence 
ocruc cure 
Score 


' Logical 
Relationships 
Score 


Course 
~ Grade 


Multiple 
Correlation 


Writing 090 
(N » 23) 


Mean 57.0 
S.D. 12.6 
r +.451 


'. 59.7 
°i ,9.1 
' VK338 


5-4,3- 
14.6 
+.376 


.61 
.50 


.451 


Writing 103* 
(N = 43,8) 


Mean 68.7 
S.D. 8.5 
r +.244 


68.7 
8.2 
+.327 


68.1 
8.5 
+.308 


2.18 
1.03 


.348 - 


Reading 102 
(N * 274) 


Mean 65.6 
S.D. 8.1 V 
r +.411 . 
. 


66,4 
+.427 


64.3 
8.4 
+.481 


2.64 
1.09 


.526 



Reading 090 is not included because no students received an unsatisfactory grade 
(U) in ,the course* 
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Table 16 

Summary of Predictive Score-Grade Correlations for 
Mathematics Courses at College A 



Course 


Computation 
Score 


Elementary 
Algebra 
Score 


Course 
Grade 


Multiple 
Correlation 


52-01 

Basic Math 
Skills 1 
(N - 90) ' 


Mean 52.2 
S.D. 7.5 
r +.434 


54.1 
4.3 
+.126 


2.01 
1.54 


.435 


52-02 

Basic Math 
Skills 2 
(N = 105) 


M&s^ 62.2 
S.D. 8.5- 
r +.399 


55.8 
5.3 
+.339 


2.29 
" 1.59 


.466 



Table 17 

Summary of Concurrent .Score-Grade Correlations for 
Mathematics Courses at College A* 



Course 


Computation 
Score 


Elementary 
Algebra 
Score 


Course 
Grade j< 


52-01 

Basic Math 
Skills 1 
(N » 78) 


Mean } 63. 3 
S.D. x 8,7 
r t +.629 


* 


• 2 - 53 A 


- 52-02 ( 
^ Basic Math J 
Skills 2 ^ 
(N » 99) 


Mean 
S.D. 

r - • 


66.5 
, 7.2 
+.626 


2.70 
1.44 

t 



The Computation test was administered as a post- 
test to only students in Basic Math Skills 1; 
Elementary Algebra was administered as a post- 
test to^only students in Basic Math Skills 2. 
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Table 18 

Summary of Predictive Score-Grade Correlations for 
Mathematics 090 Course at College B 



Course 


Computation 
Score 


Elementary * 
Algebra 
Score 


Course 
Grade 


Multiple 
Correlation 


Math 090 
(N = 461) 


Mean 67,6 
S.D. 8.7 
r +.204 


61.9 
6.4 
+.309 


.57 
.50 


.337 


✓ 




* 

Table 19 


r 




Summary of Concurrent Score-Grade Correlations for 
Mathematics 090 Course at College B 


^Course 


Computation 
Score . 


Elementary 
Algebra 
Score 


Course 
Grade 


Multiple 
Correlation 










Math 090 
(N » 393); 


Mean 68,2 
S.D, 7.5 
r +.334 


67.5 
7.9 
+.508 


V 50 


C 

.516 



50 
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APPENDIX 

CONTENT-VALIDITY QUESTIONNAIRES 
EDUCATIONAL TESTING ^ERVICE PRINCETON. N.J. 

Area Code 609 

921-9000 April 18, 1980 

CABLE.EDCCTESTSVC 
College Board Programs 



Dear College Instructor: 

As you may know, Educational Testing Service is conducting a 
study of the validity of the New Jersey, College Basic Skills Placement 
Test (NJCBSPT) for the New Jersey Basic Skills Council. One of the / 
key elements in this project is an analysis of the content of the tesc 
by New Jersey college faculty who teach beginning college freshman 
3tudents. As an Instructor of reading, writing, or mathematics for 
beginning freshmen your college, your judgment of the content of 
the test is of crucial importance both to the validation of the test, 
and to its improvement. 

Enclosed you will find a brief questionnaire as well as a copy ' 
of the test itself (Form 3BJ?) . tfould you please complete the 
questionnaire and return it in the attached envelope, with the test 
booklet, to the campus coordinator for this project whose name is on 
^the^ envelope? Please return the materials to your college coordinator 
by Friday ^pril 25 . 

Thank you in advance for your assistance in completing this 
analysis of the content of the NJCBSPT. If you have any questions, 
please feel free to calli 



Lawren ce HeclxL-^- ~ 

Associate Measurement Statistician 
Educational Testing Service 
(609) 921-9000 Ext. 3467 
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CONTENT ANALYSIS OF THE READING AND WRITING SECTIONS 
OF THE NEW JERSEY COLLEGE SASIC SKILLS PLACEMENT TEST 



In this questionnaire you are asked Co make a judgment about ten 
skills chat are intended Co be measured by Che reading and writing portions 
of che NJCBSPt. For each skill, please indicace whether it is very 
important moderately important, or unimportant xo che process of good 
reading or writing ac che beginning college freshman level.- So chac you 
nay gee a better idea of -what is msant by each skill, che item numbers 
for che icems in Form 3BJ? of che cesc chat ire incended to assess chac 
skill are also given. 



Skill Description 



Item Numbers 



Understanding main ideas 
in reading passages 



Understanding direct state- 
ments in reading passages 

Making inferences from 
reading passages 



4, Using complete sentences 



Using coordination and - 
subordination Appropriately 



" 6. 

7. 
8. 
9. 

10. 



Placing modifiers appro- 
priately 



Categorizing ideas 



Using appropriate connectives 
in sentences 

Making analogies in sentences 



Recognizing principles of organ- 
ization and relationships among 
sentences 



Reading Comprehension Items 
1, 4,7, 9, 12, 15, 18, 20, 23 ,J26', 
30,34,38,39 

Reading Comprehension Items 

2,6,11,14,16,24,29,31,32,33, 
37 

Reading Comprehension Items 
3,5,8,10,13,17,19,21,22,25, 
27,28,35,36,40 

Sentence Structure Items 
1,3,5,8,14,17,21,13,26,34 

Sentence Structure Items 

2,7,10,11,15,19,25,27,29, 

30,31,33,35 

Sentence t Structure Items 
4,6,9, 12,*13, 16,18, 20,22,24, . 
28,32 * 1 

Logical Relationships Items 
1-12 

Logical Relationships Items 
13-25 

Logical Relationships Items 
26-37 

Logical Relationships Items 
38-50 



Importance of Skill 
(circle one letter) 



u u u 

>v o 5) o a 

U £U T3 £♦ «H 

a s o s c 



A B 



ABC 



c 



ERLC 



>3 



-2 



11. The NJCBSPT also contains a 20-minute essay,. For Form 3BJP 
of the test, the essay topic and instructions are as follows:- 

,f The communities and neighborhoods we live in have various 
concerns, such as high taxes, repeated power failures, 
water pollution, the clearing of roads in winter, new 
highway construction, new parks and playgrounds, better 
street lighting,* or increased police and fire protection. 

In a well-organized essay, write abour a concern in your 
community or neighborhood, preferably one that has 
affected or may affect you. Tell what your concern is, ' 
and "why it is a concern. Indicate what action,* if any, ^ 
you or your fellow citizens have taken or plan co take to 
deal with that concern* Be sure £o be , specific." 

How important is obtaining a siting sample in assessing the writing 
ability of beginning college- students? Circle one letter below: 

A Very Important 

B Moderately Important 

C Unimportant 

12, Please specify below any skilis that are important to the reading 
and writing ability of beginning college students that are not 
assessed by : the NJCBSPT. 



13. Please complete the following information: 
Your college 




tour name and title- 



Courses- you teach (give number and name):, 



Please return this" questionnaire in the attached' envelope, with the test 
booklet, to the qampus coordinator for this* project whose name is on the 
envelQpe. Return the materials to your college coordinator by Fr«dav • 
April 25, • ' 
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CONTENT ANALYSIS OF THE MATHEMATICS SECTIONS 
OF THE NEW JE&SEZ COLLEGE BASIC SKILLS " PLACEMENT TEST 



In this questionnaire you are askad co make a judgment about eleven 
skiils that are intended tQ.be measured by the two mathematics portions 
of the NJCBSPT. The Computatioiv^Skills section (pages 23-28 in Form 3BJP 
booklet) is designed to assess students 1 proficiencies in Basic computa- 
tional skills. The -Elementary Algebra Skills section (pages 29-34 in 
booklet) is designed to assessstudents 1 proficiencies in basic algebra. 
These algebra skills inciudedB^ability to perform specific operations 
as well as the ability to ca^^Ehrough operations involving a succession 
of simple steps, . ' " % 

On the next page you are asked to judge the importance of testing each 
'of the listed skills in assessing students' proficiencies in computation 
and elementary algebra. You will be rating each skill as very important to 
assess, moderately important to assess, or unimportant to assess, 

3efor§ completing the questionnaire, please provide the following 
information: * . 



Tour. college - 



'Your name and cicle 



Courses- you teach (give number and name) 



9 



r 



55 



Skill Description- 



Item lumbers 



Computation Skills: 



Importance of Skill 
(circle one fetter) 



s * 

a 

u 

u 

o 

s 



u 

2. 



CO 

>, u 
r-r o 
<u a. 
^ 3 

*T3 



C 
u 

^» 

5 
■H 

£ 



i. 


Operations with whole numbers 


2,4 


A 


B 


C 


2. 


Operations with fractions 


1,3,6-13 




'b 


C 


3. 


Operations wish decimals 


14,16-24 . 


A 


B 


C 


_ 4. 


Solution of simple word w problems 
(involving concepts of percent, 
proportion, and average value) 


5,15,25-30 


A 


B 


C 


Elementary Algebra Skills: ' 




<> 


♦ 




1. 


Substitution in algebraic expressions 


35,36 




3 


C 


2. 


Translating English phrases into 
appropriate algebraic expressions 


45,53 


A 


> B 


C 


3. 


Operations involving algebraic 
expressions 


32,34,38,40,53,59 


A 


B 


C 


4. 


Operations involving simple 
polynomials 


31,37,41,48-52 


A 


B 


C 


5. 

* 


Operations involving exponents 
and roots 


39,42-44 


- A 


B 


C 


6, 


Solution of linear equations 


33,46,4.7,55-57,60 


A 


3 


c 


7, 


Solution of linear inequalities 


54 


A - 


3 





* 



|laase specify on the back of this page any skills that are important to the 
asse^jgp&it of computatiop and elementary algebra proficiency of ^ beginning college stu- 
dents ^that are not assessed by the NJCBSPT, * 

Please return this questionnaire in the attached envelope, with the test booklet,' 
to the campus coordinator for this project whose name is on, the envelope. .Return the 
materials to your college coordinator by Friday, April 25 . 
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CONTENT ANALYSIS OF THE MATHEMATICS SECTIONS 
OF THE NEW JERSEY COLLEGE BASIC SKILLS PLACEMENT TEST 



/ _ 

^In this questionnaire you are asked to make a judgment about eleven 
skills that are intended to be measured by the two mathematics portions - 
of the NJCBSPT. The Computation Skills section (pages 23-28 in Form 3BJP 
booklet) is designed to assess students 1 prpf iciencies in basi9 computa- 
tional skills. The Elementary Algebra Skills section (pages 2'9-34 in 
booklet) is designed to assess students* proficiencies in basic algebra. 
These algebra skills include the ability to perform specific operations 
as ve£l as the ability to cany through operations involving a succession 
of simple steps ♦ 

On the next page you are a£ked to judge the importance of testing each 
of the listed skills in assessing students 1 proficiencies in computation 
and elementary algebra* You will be rating each skill as very important to 
assess, moderately important to assess, or unimportant to assess. 

Before completing the questionnaire, please provide the following 
information; 



Your college. 



Your name and title 



Courses you teach (give number and name) : 



j 




5* 
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- Skill Description 



Item Numbers 



Importance of Skill 
(circle one letter) 



Computation Skills: 



s 

u 

o 



u 
> 



a 

qj 

«H O 

« S 

U 
O 



4J 

u 
u 
o 
a. 
S 

£ 



1. 


Operations with whole numbers 


2,4 


A 


B 


C 


2, 


Operations with fractions 


1,3,6-13 


A 


B 


C 


3. 


Operations with decimals 


14,16-24 


A 


B 


c 


4. 


Solution of simple word problems 
(involving concepts of percent, 
proportion, and average value) 


5,15,25-30 


A 


> 

B 


c 


Elementary Algebra Skills: 












• Substitution in algebraic expressions 


35,36 


A 


B 


c 


2, 


Translating English phrases into 
appropriate algebraic expressions 


45,53 


A 


3 


' c 


3. 


Operations involving algebraic 
expressions 


32,34,38,40,58,59 


A 


B 


c 


4, 


Operations involving simple 
polynomials ; . 


31,37,41,48-52 


A 


B ' 


c 


5 . 


Operations involving exponents 
and roots x 


39,42-44 


A 


3 


c 


6. 


Solution of linear equations 


33,46,47,55-57,60 


A 


B 


c 


7. 


Solution of linear inequalities 


54 


A 


B 


c 



Please specify "on the back of this page any skills that are important to the 
assessment of computation and^^lementary algebra proficiency of beginning college stu- 
dents that are not assessed by the NJCBSPT* 

Please return this questionnaire in the attached envelope, with the test booklet, 
to the campus coordinator for this project whose name is on the envelope. Return the 
materials to your college Coordinator by Friday, April 25. 



s 
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CONTENT ANALYSIS OF THE 'READING AND WRITING SECTIONS 



OF THE NEW JERSEY COLLEGE 3ASIC SKILLS PLACEMENT TEST 



In chis questionnaire you are asked co make a judgment about ten 
skills chac are inceaded co be measured by che reading and writing portions 
of che NJC3SFT* For each skill, please indicate whether it is very 

important, moderately important, or unimportant to the process of good 

reading or writing at the beginning college freshman level. So that you 
may get a better idea of what is meant by each skill, the item numbers 
for the items in Form 3BJ? of the test that are intended to assess that 
skill are also given. 



Skill Descriotion 



Item Numbers 



Importance of Skill 
(circle one letter) 



Understanding main ideas 
in reading passages 



Reading Comprehension Icems 
1,4,7,9,12,15,18,20,23,26, " 
30, 34, 38, 39 



£ V £ -J 

>* o v o a 

U O* *T3 c 

V £ OS C 



2. Understanding direct state- 
ments in reading passages 



Reading Comprehension Items 

2,6,11,14,16,24,29,31,32,33, 

37 



ABC 



3. MafcJLng inferences from 
reading passages 



Reading Comprehension. Items 
3,5,8,10,13,17,19,21,22,25, 
27,28,35,36,40 



4. Using complete sentences 



5. Using coordination and 

subordination appropriately 



6 ♦ Placing modifiers appro- 
priately 



Sentence Structure/ Items 
1,3,5,8,14,17,21,23,26,34 

Sentence Structure Items 
2,7,10,11,15,12,25,27,29, 
30,31,33,35 . 

Sentence Structure Items 

4,6,9,12,13,16,18,20,22,24, 

28,32 



Categorizing ideas 



Logical Relationships teems 
1-12 



8* Using appropriate connectives 
in sentences 



Logical Relationships Items 
13-25 



9* Making analogies in sentences 

10. Recognizing principles of organ- 
ization and relationships among 
sentences 



Logical Relationships Items 
26-37 

Logical Relationships Items 
38-50 



9 

ERLC 



59 
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11 «. fM The NJC3SPT also contains a 20-minute essay. F^r Form 3BJ? 
of che test, che essay topic and instructions are as follows: 

t 

"The communities and neighborhoods we live in have various 
concerns, such as high taxes, repeated power failures, 
watfcr pollution, the clearing of roads it winter, new 

- highway construction, new parks and playgrounds, better 
street lighting, or increased police 2nd fire protection. 

* In a well-organized essay, write aSout a poncern in your 
community or neighborhood, preferably one that has 
affected or aiay affect you, fell what your concern is, 
/ and why it is a concern. Indicate what action, if any, 
' you or your fellow citizens have taken or plan to take to 
deal with that concern- 3e sure to be specific." 

How important is obtaining a writing sample in assessing the writing 
ability of beginning college students? Circle one letter below: 

A Very Important 
B Moderately Important 
- *■ £ Unimportant * * . c * 

12. -Please specify below any skills that are important to the* reading 
and writing ability of beginning college students that are not 
assessed by the NJC3SPT. 



13. Please complete the following information: . 
Tour college ] 



Tour -name and title 



Courses you teach (give number and name) 



Please return this questionnaire in the attached envelope, with the test 
booklet, to the campus coordinator for this project whose name is on the 
envelope. Return the materials to ydur college coordinator" by Friday , 
April 25 . 
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